Papers by Frederic Thomas Kirstein
Is my Meeting Summary Good? Estimating Quality with a Multi-LLM Evaluator (2025.coling-industry)
Copied to clipboard
| Challenge: | Established metrics such as ROUGE and BERTScore have a relatively low correlation with human judgments and fail to capture nuanced errors. |
| Approach: | They propose a framework that uses a three-step assessment of individual error types, multi-agent discussion for decision refinement, and feedback-based self-training to refine error definition understanding and alignment with human judgment. |
| Outcome: | The proposed framework achieves high correlation with human judgment and a consistent rating and adaptability to custom error guidelines. |
What’s Wrong? Refining Meeting Summaries with LLM Feedback (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods for meeting summarization are limited and lack the robustness and context-based accuracy needed to maintain relevance. |
| Approach: | They propose a multi-LLM correction approach for meeting summarization using a two-phase process that mimics the human review process: mistake identification and summary refinement. |
| Outcome: | The proposed approach improves the quality of a given meeting summarization measured by relevance, informativeness, conciseness, and coherence. |